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DETAILED ACTION 
Response to Arguments 

1 . Applicant's arguments with respect to claims 52, 54-66, 68-79, 81-83, 95-99, 
111-119 have been considered but are moot in view of the new ground(s) of rejection. 

Allowable Subject Matter 

2. Claims 1 12, 1 15, and 1 18 are objected to as being dependent upon a rejected 
base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

Prior art of record fails to show or fairly suggest a sixth data structure element 
further associated with a visibility indicia indicative of whether the plurality of object in 
the video frame are enabled for being visually identified for a particular video shot. 
Kaiser teaches the sixth data structure element, but is silent on having a visibility indicia 
for enabling display. Kitsukawa teaches three modes, wherein the selection of the 
mode will determine whether to product should be displayed or prevented from being 
displayed (Kitsukawa: col. 6-7, II. 65-7). However, there is no motivation to combine the 
modes of Kitsukawa into the sixth data structure of Kaiser for presenting or preventing 
the display of content, in that the sixth data structure of Kaiser is transmitted to the user 
and the modes of Kitsukawa are local to that particular device. The examiner notes that 
the first data structure of claim 1 18 is the sixth data structure of claims 112 and 115. 



Claim Objections 
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3. Claims 54, 55, 68, 69, and 81 are objected to because of the following 
informalities: 

Claim 54 recites the limitation ""said timing data indicator" in line 2. There is 
insufficient antecedent basis for this limitation in the claim. Further claim 54 depends 
from claim 53, which is cancelled. Accordingly, for the rest of this Office Action, claim 
54 will depend from claim 52, will be interpreted as: "The system of claim 52 further 
comprising at least one timing data indicator associated with at least one of said 
plurality of data structure elements, wherein said timing data indicator indicates an 
expiration time. " 

Claim 55 recites the limitation ""said timing data indicator" in line 2, There is 
insufficient antecedent basis for this limitation in the claim. Further claim 55 depends 
from claim 53, which is cancelled. Accordingly, for the rest of this Office Action, claim 
54 will depend from claim 52, will be interpreted as: "The system of claim 52 further 
comprising at least one timing data indicator associated with at least one of said 
plurality of data structure elements, wherein said timing data indicator indicates an 
activation time. " 

Claim 68 recites the limitation ""said timing data indicator" in line 2. There is 
insufficient antecedent basis for this limitation in the claim. Further claim 68 depends 
from claim 67, which is cancelled. Accordingly, for the rest of this Office Action, claim 
68 will depend from claim 66, will be interpreted as: "The method of claim 66 further 
comprising at least one timing data indicator associated with at least one of said 
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plurality of data structure elements, wherein said timing data indicator indicates an 
expiration time. " 

Claim 69 recites the limitation ""said timing data indicator" in line 2. There Is 
insufficient antecedent basis for this limitation in the claim. Further claim 69 depends 
from claim 67, which is cancelled. Accordingly, for the rest of this Office Action, claim 
69 will depend from claim 66, will be interpreted as: "The method of claim 66 further 
comprising at least one timing data indicator associated with at least one of said 
plurality of data structure elements, wherein said timing data indicator indicates an 
activation time. " 

Claim 81 recites the limitation ""the timing data indicator" in lines 4-5. There is 
insufficient antecedent basis for this limitation in the claim. Further claim 81 depends 
from claim 80, which is cancelled. Accordingly, for the rest of this Office Action, claim 
81 will depend from claim 52, will be interpreted as: "The system of claim 52 further 
comprising at least one timing data indicator associated with at least one of said 
plurality of data structure elements, wherein [the image overlay data is associated with 
first timing data,] presentation of the image overlay is [being] synchronized with the 
video frame based on the first timing data indicator ." 

Appropriate correction is required. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 52, 55-59, 61-66, 69-73, 75-82, 95-98, 111, 113, 114, and 116 are 

rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. Patent 6,615,408 to 

Kaiser et al. (Kaiser) in view of U.S. Patent 5,889,746 to Moriyama et al. (Moriyama) 

and U.S. Patent 6,282,713 to Kitsukawa et al. (Kitsukawa). 

Regarding claims 52 and 66, Kaiser teaches placement zones (as shown in 

figure 2), which has a reference to an object in the video frame (Abstract), which 

equates to a first annotation data structure element that includes and object reference to 

an object in the video frame. Kaiser teaches a trigger zone which contains a Uniform 

Resource Identifier (URI), which equates to a second annotation data structure element 

referenced by the first identifier and including a set of data references (col. 6, II. 34-39). 

Kaiser teaches the URI being identified by the object in that the URI has a 

"<videoprod>" field which is not temporally unique into which the trigger is embedded 

(col. 6, II. 43-64, However, Kaiser is silent on the first annotation element identifying the 

second element as claimed. Moriyama teaches identifying elements via pointers, as 

shown in figure 9. Therefore, it would have been obvious to one of ordinary skill in the 

art at the time the invention was made to modify Kaiser by identifying a second element 

via a pointer as taught by Moriyama in order to efficiently manage dynamic resources. 

Further, Kaiser discloses that the image area (described using a placement zone) is 

referenced by some form of an identifier in order to link actions to the image (col. 10, II. 

1-8). Kaiser teaches a singular object in the video frame an indicating the information 
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with an HTML table (which equates to a sixth data structure element) (col. 10, II. 20-30), 
but is silent on storing image overlay data for a plurality of objects in the video frame, 
and in response to a user command visually identifying one of the plural objects. 
Kitsukawa teaches providing advertising information from numerous items in a program 
scene (fig. 5, labels 521-529, col. 8, II. 23-36), wherein in response to the advertisement 
mode selected (col. 6-7. II. 65-2) by a user, the program visually identifies at least one of 
the plurality of objects in the video frame based on the overlay data (col. 7, II. 3-20). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by storing image overlay data for a plurality of 
objects in the video frame, and in response to a user command visually identifying one 
of the plural objects as taught by Kitsukawa in order to provide an innovative and user 
friendly access to a wealth of information regarding goods and services through the 
broadcasting system (Kitsukawa: col. 4, II. 15-17). 

Regarding claims 55 and 69, Kaiser teaches placing the placement zones in the 
video stream (fig. 2), which reads on a timing data indicator associated with the at least 
on of said plurality of data structure elements. Kaiser teaches placing the placement 
zones in the video stream (fig. 2), which indicates an activation time. 

Regarding claims 56 and 70, Kaiser teaches the placement zones associated 
with a set of video frames (fig. 2, col. 6,11.1 8-33), further the trigger is associated with 
the placement zones and consequently is associated with the video program, given the 
broadest reasonable interpretation of associated. 
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Regarding claims 57 and 71 , Kaiser is silent on the elements being transmitted 
separately. Official Notice is taken separately sending data in MPEG is well known in 
the art. Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Kaiser by sending the elements separately in 
order to send data that is larger than the data packets thereby using the existing 
infrastructure to transmit packets. 

Regarding claims 58 and 72, Kaiser teaches a trigger which is used to display 
the action being performed fig. 6A-6b, which reads on the first set of annotation 
references including a data field. However, Kaiser is silent on a second identifier 
referencing a third annotation data structure element. Official Notice is taken that the 
use of pointers is well known in the art. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser to implement 
a pointers (which reads on a reference) in order to access another portion of a data 
structure thereby providing a system that efficiently manages variable length data 
structures and increasing the robustness of the system. 

Regarding claims 59 and 73, Kaiser is silent on the annotation data field is a title 
data field and the third data structure element is a string including the title of the object. 
Moriyama teaches a pointer to a text string wherein the text string can be the title (col. 
15-16, II. 65-31; see also fig. 9). Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by pointing 
to a string including the title as taught by Moriyama in order to efficiently manage 
variable length fields and manage the memory of the system. 
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Regarding claims 61, 62, 75. and 76, Kaiser is silent on a variable paranneter 
field and a variable value. Official Notice is taken that a variable parameter field and 
variable values are well known in the art. Therefore, it would have been obvious to one 
of ordinary skill in the art at the time the invention was made to modify Kaiser by using a 
variable parameter field and variable values in order to efficiently manage the memory 
by dynamically referencing and allocating the memory. 

Regarding claims 63 and 77, Kaiser is silent on never duplicating the first and 
second identifiers. Official Notice is taken that never duplicating identifiers is well 
known in the art. Therefore, it would have been obvious to one of ordinary skill in the 
art at the time the invention was made to modify Kaiser by never duplicating identifiers 
in order to appropriately reference the appropriate information thereby eliminating 
access to the incorrect data structure. 

Regarding claims 64 and 78, Kaiser teaches location and shape information, 
which equates to a sixth data structure element (fig. 6B, col. 6, II. 20-41). 

Regarding claims 65 and 79, Kaiser teaches that the sixth data structure is 
associated with a video frame (see fig. 2). 

Regarding claims 80 and 96, Kaiser teaches a visual highlight of an image as an 
overlay (see fig. 6B, label 6500, col. 10, II. 20-41), which is in the form of an HTML 
table, which equates to a third data structure storing image overlay data, but is silent on 
displaying in response to a user command. Official Notice is taken that customizing 
interfaces based on user preferences is well known in the art. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
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modify Kaiser by customizing the interfaces in order to control the highlighted portions 
of Kaiser thereby enabling the user to further customize their viewing environnnent. 

Regarding claims 81 and 97, Kaiser teaches the image overlay data is 
associated with time data and synchronized to the video frames (col. 10, IL 20-41). 

Regarding claims 82 and 98, Kaiser teaches identifying the object (col. 1 0, II. 20- 
41 , fig. 6B), but Kaiser is silent on determining the difference from the a first time and 
the current time to initiate the event. Official Notice is taken that determining the 
difference in time to perform an event is well known in the art. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kaiser by determining the difference in time to perform an event in order to 
efficiently manage events thereby reducing the processing of the system by selectively 
querying the system clock. 

Regarding claim 95. Kaiser teaches a video production (1100), which reads on a 
first processor for creating the data structures, such as a DVD (col. 5, II. 1-7). Kaiser 
teaches placement zones (as shown in figure 2), which has a reference to an object in 
the video frame (Abstract), which equates to a first annotation data structure elennent 
that includes and object reference to an object in the video frame. Kaiser teaches a 
visual highlight of an image as an overlay (see fig. 6B, label 6500, col. 1 0, 11. 20-41 ), 
which is in the form of an HTML table, which equates to a third data structure, which 
reads on the third data structure. Kaiser teaches a trigger zone which contains a 
Uniform Resource Identifier (URI), which equates to a second annotation data structure 
element referenced by the first identifier and including a set of data references (col. 6, II. 
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34-39). Kaiser teaches the URI being identified by the object in that the URI has a 
"<videoprod>" field which is not temporally unique into which the trigger is embedded 
(col. 6, II. 43-64. However, Kaiser is silent on the first annotation element identifying the 
second element and third element as claimed. Moriyama teaches identifying elements 
via pointers, as shown in figure 9. Therefore, it would have been obvious to one of 
ordinary skill in the art at the time the invention was made to modify Kaiser by 
identifying a second and third element via a pointer as taught by Moriyanna in order to 
efficiently manage dynamic resources. Further, Kaiser discloses that the image area 
(described using a placement zone) is referenced by some form of an identifier in order 
to link actions to the image (col. 10, II. 1-8). Kaiser teaches the image overlay data is 
associated with time data and synchronized to the video frames (col. 1 0, II. 20-41). 
Kaiser teaches a singular object in the video frame an indicating the information with an 
HTML table (which equates to a sixth data structure element) (col. 10, II. 20-30), but is 
silent on storing image overlay data for a plurality of objects in the video frame, and in 
response to a user command visually identifying one of the plural objects. Kitsukawa 
teaches providing advertising information from numerous items in a program scene (fig. 
5, labels 521-529, col. 8, II. 23-36), wherein in response to the advertisement mode 
selected (col. 6-7, II. 65-2) by a user, the program visually identifies at least one of the 
plurality of objects in the video frame based on the overlay data (col. 7, II. 3-20). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by storing image overlay data for a plurality of 
objects in the video frame, and in response to a user command visually identifying one 
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of the plural objects as taught by Kitsukawa in order to provide an innovative and user 
friendly access to a wealth of information regarding goods and services through the 
broadcasting system (Kitsukawa: col. 4, II. 15-17). 

Regarding claims 111, 114, and 117, Regarding claims 53 and 67, Kaiser 
teaches placing the placement zones in the video stream (fig. 2), which reads on a 
timing data indicator in association with one of the data structure elements, which 
synchronizes the display of the mask with the video frame based on the retrieved time 
data indicator. 

Regarding claim 113, 116, and 1 19, Kaiser is silent on a plurality of objects being 
identified in sequence in response to one or more user commands. Kitsukawa teaches 
cursor direction keys for controlling cursor movement on the display (col. 4, II. 46-54) for 
navigating to plural objects in the display (fig. 5, labels 521-529, col. 8, II. 23-36), 
however both Kaiser and Kitsukawa are silent on the objects are visually identified in 
sequence in response to one or more user commands. Official Notice is taken that 
sequencing through objects in response to user commands is well known, such as in 
the analogous personal computer art of using "tab" to navigate around a GUI or in the 
television art of using a cursor to navigate through menus. Therefore, it would have 
been obvious to one of ordinary skill in the art at the time the invention was made to 
modify Kaiser and Kitsukawa by as taught by visually identifying objects in sequence in 
order to effectively provide a means to the user to select a device or desired product 
thereby simplifying the selection process. 
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6. Claims 54, 68, 83, and 99 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over U.S. Patent 6,615,408 to Kaiser et al. (Kaiser). U.S. Patent 5,889,746 
to Moriyama et al. (Moriyama) and U.S. Patent 6,282,713 to Kitsukawa et al. 
(Kitsukawa) in view of U.S. Patent 6,415,438 to Blackketter et al. (Blackketter). 

Regarding claims 54 and 68, Kaiser teaches placing the placement zones in the 
video stream (fig. 2), which reads on a timing data indicator associated with the at least 
on of said plurality of data structure elements. Kaiser is silent on indicating an 
expiration time. Blackketter teaches expiring triggers (col. 3, II. 13-22, col. 10, II. 24-35). 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser by indicating an expiration time in order to ignore 
invalid triggers (Blackketter: col. 10, II. 24-35). 

Regarding claims 83 and 99, Kaiser is silent on indicating an expiration time. 
Blackketter teaches expiring triggers, which are indicative of the last instance the data 
structure is used (col. 3, II. 13-22, col. 10, II. 24-35). Therefore, it would have been 
obvious to one of ordinary skill in the art at the time the invention was made to modify 
Kaiser by indicating an expiration time in order to ignore invalid triggers (Blackketter: 
col. 10, II. 24-35). Kaiser and Blacketter are silent on removing the data structures. 
Official Notice is taken that removing expired information is well known in the art. 
Therefore, it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to modify Kaiser and Blacketter by removing the data structures in 
order to conserve memory and efficiently manage the memory resources. 
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Conclusion 



Any inquiry concerning tiiis communication or earlier communications from the 
examiner should be directed to Andrew Y Koenig whose telephone number is (703) 
306-0399. The examiner can normally be reached on M-Th (7:30 - 6:30). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Christopher Grant can be reached on (703) 305-4755, The fax phone 
number for the organization where this application or proceeding is assigned is 703- 
872-9306. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-21 7-91 97 (toll-free). 
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